Revisiting Near-collinearity: Numerical vs. Statistical Perspectives

The primary objective is to bring out and discuss the issues raised by the differences between the numerical and the statistical perspective on nearcollinearity in the context of the Linear Regression model. The numerical perspective views the problem as stemming from the ill-conditioning of the (X|X) matrix, which does not necessarily originate in high sample correlations among the regressors. The latter, however, constitutes the cornerstone of the statistical perspective. The numerical measures, such as the condition number, are applicable to an arbitrary matrix of numbers, irrespective of whether these numbers denote statistical data or not. On the other hand, the various statistical measures invoke probabilistic assumptions pertaining to the stochastic process underlying the particular data. When these assumptions are invalid, the statistical measures can be highly misleading. The paper argues that these differences between the numerical and statistical measures of ill-conditioning have important implications that call into question certain aspects of the current conventional wisdom on near-collinearity. The proposed perspective views the near-collinearity as data inadequacy in learning about the parameters of interest. This perspective is used to call into question the pertinence of VIFs and shed light on certain issues raised by the literature on centering the data.